{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 程序说明\n",
    "时间：2016年11月16日\n",
    "\n",
    "说明：该程序是一个包含两个卷积层、一个池化层和一个全连接层组成的神经网络。\n",
    "\n",
    "数据集：MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.加载keras模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "np.random.seed(1337)  # for reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "from keras.utils import np_utils\n",
    "from keras import backend as K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 如需绘制模型请加载plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.utils.vis_utils import plot_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.变量初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 128 \n",
    "nb_classes = 10\n",
    "nb_epoch = 12\n",
    "\n",
    "# 输入数据的维度\n",
    "img_rows, img_cols = 28, 28\n",
    "# 使用的卷积滤波器的数量\n",
    "nb_filters = 32\n",
    "# 用于 max pooling 的池化面积\n",
    "pool_size = (2, 2)\n",
    "# 卷积核的尺寸\n",
    "kernel_size = (3, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.准备数据\n",
    "需要将数据还原成28*28大小的数组\n",
    "\n",
    "注意：Theano和TensorFlow的维度表示含义是不一样的。\n",
    "\n",
    "'th'模式，也即Theano模式会把100张RGB三通道的16×32（高为16宽为32）彩色图表示为下面这种形式（100,3,16,32），Caffe采取的也是这种方式。第0个维度是样本维，代表样本的数目，第1个维度是通道维，代表颜色通道数。后面两个就是高和宽了。而TensorFlow，即'tf'模式的表达形式是（100,16,32,3），即把通道维放在了最后。这两个表达方法本质上没有什么区别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "# the data, shuffled and split between train and test sets\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "if K.image_dim_ordering() == 'th':\n",
    "    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)\n",
    "    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)\n",
    "    input_shape = (1, img_rows, img_cols)\n",
    "else:\n",
    "    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)\n",
    "    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)\n",
    "    input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "X_train = X_train.astype('float32')\n",
    "X_test = X_test.astype('float32')\n",
    "X_train /= 255\n",
    "X_test /= 255\n",
    "print('X_train shape:', X_train.shape)\n",
    "print(X_train.shape[0], 'train samples')\n",
    "print(X_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转换类标号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# convert class vectors to binary class matrices\n",
    "Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, nb_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.建立模型\n",
    "### 使用Sequential（）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(nb_filters, (kernel_size[0], kernel_size[1]),\n",
    "                        padding='valid',\n",
    "                        input_shape=input_shape))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Conv2D(nb_filters, (kernel_size[0], kernel_size[1])))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=pool_size))\n",
    "model.add(Dropout(0.25))\n",
    "\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(nb_classes))\n",
    "model.add(Activation('softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 打印模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_7 (Conv2D)            (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "activation_13 (Activation)   (None, 26, 26, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_8 (Conv2D)            (None, 24, 24, 32)        9248      \n",
      "_________________________________________________________________\n",
      "activation_14 (Activation)   (None, 24, 24, 32)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_4 (MaxPooling2 (None, 12, 12, 32)        0         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 12, 12, 32)        0         \n",
      "_________________________________________________________________\n",
      "flatten_4 (Flatten)          (None, 4608)              0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 128)               589952    \n",
      "_________________________________________________________________\n",
      "activation_15 (Activation)   (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 10)                1290      \n",
      "_________________________________________________________________\n",
      "activation_16 (Activation)   (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 600,810\n",
      "Trainable params: 600,810\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制模型结构图，并保存成图片"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "ename": "ImportError",
     "evalue": "Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-21-4171ccb20a6d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mplot_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mto_file\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'model-cnn.png'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/opt/anaconda/lib/python2.7/site-packages/keras/utils/vis_utils.pyc\u001b[0m in \u001b[0;36mplot_model\u001b[0;34m(model, to_file, show_shapes, show_layer_names, rankdir)\u001b[0m\n\u001b[1;32m    129\u001b[0m             \u001b[0;34m'LR'\u001b[0m \u001b[0mcreates\u001b[0m \u001b[0ma\u001b[0m \u001b[0mhorizontal\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    130\u001b[0m     \"\"\"\n\u001b[0;32m--> 131\u001b[0;31m     \u001b[0mdot\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel_to_dot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshow_shapes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshow_layer_names\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrankdir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    132\u001b[0m     \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mextension\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplitext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mto_file\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    133\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mextension\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda/lib/python2.7/site-packages/keras/utils/vis_utils.pyc\u001b[0m in \u001b[0;36mmodel_to_dot\u001b[0;34m(model, show_shapes, show_layer_names, rankdir)\u001b[0m\n\u001b[1;32m     50\u001b[0m     \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSequential\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     51\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 52\u001b[0;31m     \u001b[0m_check_pydot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     53\u001b[0m     \u001b[0mdot\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpydot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     54\u001b[0m     \u001b[0mdot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'rankdir'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrankdir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda/lib/python2.7/site-packages/keras/utils/vis_utils.pyc\u001b[0m in \u001b[0;36m_check_pydot\u001b[0;34m()\u001b[0m\n\u001b[1;32m     25\u001b[0m         \u001b[0;31m# pydot raises a generic Exception here,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     26\u001b[0m         \u001b[0;31m# so no specific class can be caught.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 27\u001b[0;31m         raise ImportError('Failed to import pydot. You must install pydot'\n\u001b[0m\u001b[1;32m     28\u001b[0m                           ' and graphviz for `pydotprint` to work.')\n\u001b[1;32m     29\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mImportError\u001b[0m: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work."
     ]
    }
   ],
   "source": [
    "plot_model(model, to_file='model-cnn.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 显示绘制的图片\n",
    "![image](http://p1.bpimg.com/4851/4c386bd0784d9b4d.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.训练与评估\n",
    "### 编译模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='adadelta',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/12\n",
      "60000/60000 [==============================] - 4s - loss: 0.0493 - acc: 0.9856 - val_loss: 0.0314 - val_acc: 0.9885\n",
      "Epoch 2/12\n",
      "60000/60000 [==============================] - 4s - loss: 0.0460 - acc: 0.9861 - val_loss: 0.0297 - val_acc: 0.9892\n",
      "Epoch 3/12\n",
      "60000/60000 [==============================] - 4s - loss: 0.0449 - acc: 0.9869 - val_loss: 0.0293 - val_acc: 0.9889\n",
      "Epoch 4/12\n",
      "60000/60000 [==============================] - 4s - loss: 0.0430 - acc: 0.9875 - val_loss: 0.0296 - val_acc: 0.9895\n",
      "Epoch 5/12\n",
      "37120/60000 [=================>............] - ETA: 1s - loss: 0.0421 - acc: 0.9879"
     ]
    }
   ],
   "source": [
    "model.fit(X_train, Y_train, batch_size=batch_size, epochs=nb_epoch,\n",
    "          verbose=1, validation_data=(X_test, Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print('Test score:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  },
  "ssap_exp_config": {
   "error_alert": "Error Occurs!",
   "initial": [],
   "max_iteration": 1000,
   "recv_id": "",
   "running": [],
   "summary": [],
   "version": "1.1.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
